Branch-and-Bound and Beyond

The branch-and-bound algorithm is stated in general i ty, and i l l us t ra ted by two applications, unidirect ional graph search, and reducing a sparse matrix to i t s minimal band form. The algorithm is then generalised to mult iple par t i t ions , applied to bid i rect ional graph searching for both heurist ic and non-heuristic searches, and further extended to graph searches and problem solving with subgoals. 641 1.0 Introduction and Summary The branch-and-bound algorithm is a simple technique for the optimisation search for the minimum (or maximum) of a funct ion. It is very basic to a r t i f i c i a l intel l igence and operations research, and has been repeatedly rediscovered and used in the solution of hard problems. (1,2,6,9) In th is paper I shal l give a general statement of the algorithm, and then consider the application to reducing a sparse matrix to the band matrix of minimal band-width. After th is I shall show that the well known graph searching methods (4,5,9) are somewhat disguised examples of branchand-bound. This simple branch-and-bound idea w i l l then bp extended to a more complex optimisation search with multiple star t ing points, in which form it w i l l be used to solve the bid i rect ional graph-search problem, giving a new unif ied view of exist ing solutions, both heurist ic and non-heuristic. F inal ly the application of multiple branch-and-bound is further extended to cover the use of subgoals in graph search and problem solving. Graph searching as a problem area in a r t i f i c i a l intel l igence is important because many problems can be represented as a graph which requires a path to be found between two given nodes (3,5,9) . Any path between the start and termination nodes would constitute a solut ion, but a path of minimal length (= number of steps) can be thought of as the most 'elegant' solut ion, and is often aimed fo r . Hence icneral Problem Solving becomes f inding the shortest path in a graph, which in turn i s , as we shal l see, a special application of the branch-and-bound al Torithm. However, not. a l l problems are most natural ly formulated as graph searches; for example the t rave l l ing salesman problem is expressible as a shortest path problem, but occurs more natural ly as a branch-andbound application (6). Another example is a matching problem in chromosome analysis considered by Montonari (7); the natural way to tackle this problem is d i rect ly using branch-and-bound, but the author laboriously converts the problem to a graph search and then applies the algorithm of Hart, Nilson and Raphael (5). 642 Session No. 15 Heuristic Problem Solving The actual lower bounding function g depends upon the part icular problem and In that sense is a 'heurist ic funcztion'. The more information about the problem that is used, the closer g approaches the greatest lower bound g*, the more e f f ic ient is the algorithm as outlined above. One could even contemplate using any estimate of the greatest lower bound, recognising that if the bounding condition is v io lated, a non-optimal solution may be returned, but the error could be no more than the error in the estimator. In the form given, the algorithm may only f ind a candidate solution point after considerable searching and branching: in practice this is undesirable since it may be necessary to terminate search early with a suboptimal solut ion, because of exceeding either storage or time l im i t s . This requires a modification to the basic algorithm to force the search to refine part i t ions un t i l a possible solution point has been found and then to continue searching un t i l the minimal point is found; one way to do this would be to subtract a depth factor from the bounds to bias the search towards depth. An additional advantage to locating a potential solution point early in the search is that pruning of the search tree is possible, while we acquire the disadvantage of (possibly) more i tera t ions. Note however that we are s t i l l guaranteed f inding the optimum providing that early termination is not forced upon us. 2.1 Application to Band Matrix Reduction A sparse matrix is one in which the Session No. 15 Heuristic Problem Solving 643 majority of the elements are zero: these arise in many problems such as engineering structural analysis.(10) Storing sparse matrices in the usual way is wasteful; one solution is to store only the non-zero elements, for example by using hashing methods.(6) This leads to special numerical techniques.(10) A preferred solution would be to reduce the matrix by row and column permutations to a band matrix ( i . e . a matrix in which a l l non-zero elements l i e in some band close to the diagonal), for which only the band elements need be stored: the advantage is then that standard and ef f ic ient numerical techniques are applicable. The reduction to a band matrix is normally carried out by ad hoc methods, by sk i l led humans pr ior to computer runs: what is desired is a guaranteeable method for performing this reduction. The branchand-bound algorithm proved ideal . For an n by n matrix, there are (n!) possible configurations, and exhaustive searches are untenable. The technique used was to par t i t ion by row or column selection. Assuming for the moment that the lower bound function g is available (this w i l l be outlined la te r ) , then the f i r s t par t i t ion of the t o t a l i t y of possible configurations is to consider the n possible selections of part icular columns of the or ig inal matrix for the f i r s t posit ion in the reduced matrix. Among these n subsets, the one with smallest lower bound is selected, and the n possible selections of rows for the f i r s t posit ion of the reduced matrix considered. Among the result ing 2n-l subsets that with minimal lower bound is selected for row or column par t i t ion as appropriate, and so on, for a minimum of 2n par t i t ions . The lower bound function computes a best possible bandwidth for a par t ia l l y determined matrix (with the f i r s t m columns fixed and the f i r s t m or m-1 rows fixed) as the maximum of the following( i ) for the fixed portion at top l e f t ; count from the diagonal of the fixed portion along the rows and columns to locate the last non-zero.element; within the fixed portion (m by m or m by m-1) the elements are f ixed, while beyond the fixed portion it is assumed that the non-zero elements could be positioned immediately following the fixed columns or rows. Repeat this for each row and column within the fixed port ion, taking the maximum count found. ( i i ) for the unfixed portion at bottom r igh t ; count the number of elements in each row and column and assume that these could be placed symmetrically about the diagonal hence bound to bandwidth here is maximum of [(count + 1)/2] where [I] means the integ